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Overview 


Maturity  of  Assurance  Cases 

ISO  15026-2  Assurance  Case  Standard 

Goal  Structured  Notation 

Example  from  Industry 

Confidence  Work  at  the  SEI 

Other  Current  Work  on  Assurance  Cases 

Closing  Thoughts 
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Maturity  of  Assurance  Case  Technology 


Developed  in  late  90s  in  Europe 

Used  for  safety  cases  in  Europe  for  over  20  years 

The  UK  Ministry  of  Defence  requires  generation  of  a  compelling  case  to 
support  claims  that  specific  safety  requirements  are  met: 

“The  safety  case  shall  consist  of  a  structured  argument,  supported  by  a 
body  of  evidence,  that  provides  a  compelling,  comprehensible  and  valid 
case  that  a  system  is  safe  for  a  given  application  in  a  given  operating 
environment.”  [DEFSTAN  00-56  (Part  1)/4] 

ISO  standard  under  development  (ISO  15026-2) 

NRC  Report:  “Software  for  Dependable  Systems:  Sufficient  Evidence?” 
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ISO/IEC  15026-2:  Assurance  Case 


INTERNATIONAL  ISO/IEC 

STANDARD  15026-2 


First  edition 
2011-02-15 


Systems  and  software  engineering  — 
Systems  and  software  assurance  — 

Part  2: 

Assurance  case 


Reference  number 
ISO/IEC  15026-2:201 1(E) 

©ISO/IEC  2011 


/r  r\ 

The  assurance  case  is 
to  be  delivered  and 
maintained  with  the 
system 


Claim:  A  proposition  to  be  assured  (e.g.,  “The 
system  is  safe”) 

Evidence:  A  fact,  datum,  object,  claim,  or  other 
assurance  case 

Argument:  A  reason  why  the  set  of  evidence 
shows  that  the  claim  is  true 

Justification:  A  reason  why  a  claim  has  been 
chosen 

Assumption:  A  claim  that  appears  as  evidence 

An  Assurance  Case  is  a  quadruple  a=(c,j,es,g) 
where  c  is  a  claim,  j  is  a  justification,  es  is  a  set 
of  evidence,  and  g  is  an  argument  which 
assures  c  using  es. 

This  definition  is  recursive 
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Goal  Structuring  Notation  (GSN)  -  Kelly  1998 


A  specific  notation  for  an  assurance  case  consistent  with  15026-2. 


Developed  to  help  organize  and  structure  safety  cases  in  a  readily 
reviewable  form 

Used  successfully  for  over  a  decade  to  document  safety  cases  for  aircraft 
avionics,  rail  signaling,  air  traffic  control,  and  nuclear  reactor  shutdown 


Shows  how  claims 


and  eventually  supported  by  evidence 
while  making  clear  the  argumentation  strategies 


are  broken  down  into  sub-claims, 

or 


O 


adopted, 


the  rationale  for  the  approach  (assumptions,  justifications) 

CD 


A/J 


and  the  context 


in  which  claims  are  stated 
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Example:  Battery  Exhaustion  -  Part  One 
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Example:  Battery  Exhaustion  -  Part  Two 


C4 

When  operating  on  battery  power 
visual  and  auditory  alarms  are 
launched  at  least  {x}  minutes 
prior  to  battery  exhaustion  but  no 
more  than  {x+y}  minutes  prior 


C2 

The  battery  exhaustion 
hazard  has  been  adequately 
mitigated 


C3 

Caregiver  is  notified 
sufficiently  soon  (but  not 
too  soon)  prior  to  battery 
exhaustion 


Argue  over  hazards 
causing  failure  to  notify 
caregiver  in  a  timely 
manner 


C5 


Visual  and  auditory  alarms  are 
loud  enough  to  be  heard  and 
identified  in  the  anticipated 
clinical  setting 


C6 

{x}  minutes  warning  prior  to 
battery  exhaustion  is 
sufficient  time  to  allow 
corrective  action  in  the 
anticipated  clinical  setting 


A  late  warning  won't  give 
the  caregiver  time  to  stop 
current  activities  and  plug 
the  pump  in.  An  early 
warning  maybe  ignored. 
This  depends  on  the 
clinical  setting. 


Cx2 

Caregiver  doesn't 
notice  alarm;  amount 
of  warning  time  too 
little  for  the  anticipated 
clinical  setting 
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02  The  Task 


We  were  asked  to  assure  the  safety  of  a  system  for  guiding 
aircraft  onto  ships  in  bad  weather.  This  was  to  consider  the 
whole  ship/equipment/aircraft  system  of  systems,  taking  into 
account: 

•  Human  factors. 

•  The  operating  environment. 

•  Operating  procedures. 

•  Maintenance  &  Management. 

An  Operational  Safety  Case  (OSC)  was  needed. 


QinetiQ 


Extracted  from  “Assuring  Operational  Systems  — A  Safety  Case  Study”  by 
Simon  Di  Nucci,  Systems  and  Software  Technology  Conference,  2008 


05  Approach  -  OSC  Safety  Strategy 


QinetiQ 


Extracted  from  “Assuring  Operational  Systems  — A  Safety  Case  Study”  by 
Simon  Di  Nucci,  Systems  and  Software  Technology  Conference,  2008 


What  confidence  should  be  placed  on  an  AC? 


Given  the  evidence,  how  confident  should  we  be  in  the  claim  Cl?  Why? 
What  does  it  mean  to  have  confidence  in  the  claim? 

What  could  be  done  to  improve  confidence?  Why? 
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The  Basis  for  Confidence  in  a  Claim 


A  classic  philosophical  problem: 

•  Justify  belief  in  a  hypothesis 
Use  Induction 

•  Enumerative:  Support  increases  as  confirming  instances  are  found 


Using  past  experience  as  the  basis  for  predicting  future  behavior 
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Eliminative  Induction 


Support  for  a  claim  increases  as  reasons  for  doubt  are  eliminated 
CLAIM:  The  light  turns  on  (when  the  switch  is  flicked). 


Bulb  OK?  Power?  Wired? 


Confidence  increases  as  doubts  are  eliminated 
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What  confidence  should  be  placed  on  an  AC? 


How  confident  in  Cl?  Why?  (Number  of  uneliminated  doubts) 

What  does  it  mean  to  have  confidence?  (Lack  of  doubt) 

What  could  be  done  to  improve  confidence?  Why?  (Elim.  more  doubts) 
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A  Small  Example 


Inference  rule  (IR) 
for  validating  a 
claim 


Undercutting  defeaters  (UC) 
attack  rule  sufficiency 


If  these  reasons 
are  not  true,  then 
the  evidence  is 
valid 


Key  Ideas 


Confidence  grows  as  doubts  are  identified  and  eliminated 

•  Doubts  about  a  claim  (rebutting  defeater) 

-  Why  claim  may  be  invalid 


R2.1 

Unless  there  is 
unrestricted  user 
input  to  a  query 


•  Doubts  about  evidence  (undermining  defeater) 
-  Why  evidence  may  be  invalid 


UM4.1 


But  the  evidence  is 
based  upon  faulty 
sanitation  rules 

\ _ S 


•  Doubts  about  reasoning  (undercutting  defeater) 
-  Premise  ok;  conclusion  uncertain 


UC3.3 

Unless  there  is 
another  way 
sql  injection 

could  occur 
^ _ __ _ / 
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Other  State  of  the  Art 


John  Knight  -  University  of  Virginia 

•  Confidence  cases:  a  confidence 
argument  created  in  parallel  to  the  safety 
argument  that  documents  the  confidence 
in  the  structure  and  basis  of  the  safety 
argument. 

Tim  Kelly  -  University  of  York 

•  Evidence  elaboration:  modeling  evideno 
to  better  understand  it  and  its  evaluation 
for  the  purpose  of  explicit  integration  of 
the  source  data  of  evidence  and  the 
safety  case  argument. 


Safety  Claim 
(Goal* 


Ui=  SupportetfBy 


EvidencsX 

ResuftAssertoo 

|  tGoafl 


Lk?:  SupportodDy 


X 


ItemX 
Referer 
[Solution*  / 


Item) 
i  Reference 
\(SolL 


Sauce  Data  at 

Source  Data  of 

Evidence  ItemX 

Evidence  Item  ¥ 
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Concluding  Thoughts 


This  has  been  a  quick  overview  of  assurance  cases  and  confidence  and 
an  introduction  to  the  concept  eliminative  argumentation  as  developed 
by  the  SEI. 

•  It  is  not  a  comprehensive  review  of  all  that  is  happening  in  the  area. 

•  The  SEI  has  been  applying  Baconian  probabilities  to  confidence  maps  to 
show  how  much  different  portions  of  the  argument  contribute  to  overall 
confidence  -  something  that  may  prove  useful  for  incremental  certification. 

Assurance  cases  have  been  proven  effective  in  the  safety  domain. 

•  The  effectiveness  of  confidence  cases  and  eliminative  induction  have  yet  to 
be  demonstrated  in  practice. 


_ —  Assurance  Cases 

=1E-  Software  Engineering  Institute  Carnegie  Mellon  University  Charles  b.  weinstock,  January  2015  i8 

*  ©  2015  Carnegie  Mellon  University 


Contact  Information 


Charles  B.  Weinstock 

Principal  Researcher 
Software  Solutions  Division 
Telephone:  +1  412-268-7719 
Email:  weinstock@sei.cmu.edu 


U.S.  Mail 

Software  Engineering  Institute 
4500  Fifth  Avenue 
Pittsburgh,  PA  15213-2612 
USA 
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